1800.0 - Australian Marriage Law Postal Survey, 2017  
ARCHIVED ISSUE Released at 11:30 AM (CANBERRA TIME) 15/11/2017  First Issue
   Page tools: Print Print Page Print all pages in this productPrint All

QUALITY AND INTEGRITY STATEMENT

Summary

Australian Marriage Law Postal Survey (AMLPS) statistics meet the requirements of the Census and Statistics (Statistical Information) Direction 2017 issued by the Treasurer on 9 August 2017.

The collection and processing of survey responses was undertaken with rigour and the results accurately reflect the views of survey respondents.

An overall participation rate of 79.5%, with consistent distribution across age groups, gender and geography, is a strong indicator of quality. This high participation rate was achieved because of Australians' significant interest in this topic combined with the statistical collection design and process which promoted participation and ensured quality and integrity, in particular:

    • A simple survey form containing a single question, supported with straightforward instructions that made it easy for participants to understand and respond.
    • The very high proportion of participants complying with the form's instructions, enabling accurate coding of responses (with accuracy reviewed by external observers).
    • Rigorous survey methods that included quality controls and integrity checks which were subject to independent review and assurance.
    • Protections against fraud that included mechanisms to guarantee only one response was counted for each participant.
    "The approach adopted by the ABS was innovative, thorough and comprehensive and has supported the transparent and rigorous management of risks to the Survey."

    Protiviti


    Protiviti was engaged to provide independent assurance of the rigour and quality of survey processes.

Data quality declaration

ABS Quality Declarations provide information about the quality of statistics to promote informed and appropriate use.

Institutional environment
Definition: Information on the institutional environment within which the data are collected and processed. This includes such information as legislative obligations, financing and governance, and the transparency of the operations.

The ABS is Australia's national statistical agency, providing trusted official statistics on a wide range of economic, social, population and environmental matters of importance to Australia. The ABS plays an important leadership role coordinating statistical activities and collaborating with official bodies in the collection, compilation, analysis and distribution of statistics.

More information on the ABS institutional environment can be found within the Methods, Standards and Classifications page of the ABS website.

Recognising many in the community expected the ABS to operate a plebiscite-like process, the ABS included a number of special integrity measures in the conduct of the AMLPS:
    • independent assurance of quality and integrity controls
    • an 'external observer' program through which observers nominated by two ('Yes' and 'No') committees of parliamentarians reviewed the way ABS coded the marks on more than 600,000 survey form responses
    • a Fraud Control Plan and fraud control measures.

More detail on these measures is provided in the Integrity Section below.

Relevance
Definition: Information that will allow users to determine whether the data presented is relevant for a particular need
The AMLPS was conducted to meet the requirements of the Census and Statistics (Statistical Information) Direction 2017issued by the Treasurer on 9 August 2017 (and amended on 17 August 2017). ABS was directed to collect statistical information about:
    • the proportion of electors who wish to express a view about whether the law should be changed to allow same-sex couples to marry
    • the proportion of participating electors who are in favour of the law being changed to allow same-sex couples to marry
    • the proportion of participating electors who are against the law being changed to allow same-sex couples to marry.

AMLPS statistics meet the requirements of the Government's direction.

Timeliness
Definition: Information that will help users to determine if the data is current enough for the intended purpose, or if it relates to the period of interest.
The statistical collection occurred from 13 September 2017 to 7 November 2017, with the results released on 15 November 2017.

Accuracy
Definition: Information about how precisely the statistics measure the true quantities of interest
79.5% of eligible Australians responded to the survey. This high level of participation was consistently distributed across:
    • age groups (with all age group participation rates above 71.9%)
    • gender (male participation rate of 77.3% and female participation rate of 81.6%)
    • states and territories (7 out of 8 states and territories with participation rates over 77.9%)
    • divisions (146 of 150 Federal Electoral Divisions with participation rates above 70.5%).
    The ABS implemented a range of measures to support participation, including:
      • an extensive communication campaign
      • the ability to use a trusted person to assist with survey form completion
      • paperless options where post wouldn't work
      • easy replacement of survey forms
      • pick-up and drop-off locations in capital cities and regional and remote areas
      • extra support to promote participation by:
            • Remote Aboriginal and Torres Strait Islanders
            • Australians overseas including posted Australian Government employees and those living in Australian Territories
            • People who were away from home (i.e. long term domestic travellers)
            • People with a disability
            • People who do not speak English
            • People experiencing homelessness
            • People in aged care facilities
            • Silent electors
            • People serving custodial sentences (of less than 3 years).

    "The ABS proactively engaged with the disability sector in order to determine best practice supports... appropriate adjustments were also made available within a relatively short period of time. In my view, this set the Bureau apart as a leader in implementing accessible alternatives on a national scale."

    Alistair McEwin 
    Disability Discrimination Commissioner

    Participation was promoted through clear communication of ABS' commitment to keeping survey responses completely confidential. A Privacy Plan was published on the marriage survey web site and privacy measures were validated by an independent expert.

    The form and accompanying letter were designed using behavioural economics principles, then tested and refined to ensure both comprehension and easy, early completion.

    Coding of survey responses was highly automated with 95.0% of survey forms automatically coded. The remaining forms were manually coded by trained staff. Feedback from external observers confirmed that all coding was highly accurate. Further information is available on the External Observer Program page.

    Coherence
    Definition: Information about the comparability of the information with other key related statistics, and changes in this collection over time

    This collection has not been conducted previously, and no other source of these data exists.

    The Australian Electoral Commission (AEC) Federal Electoral Divisions used in the AMLPS concord well with the Commonwealth Electoral Divisions usually used by the ABS. Further information is available in the Australian Statistical Geography Standard (ASGS).

    The age classification is consistent with other ABS collections. The gender data item on the Commonwealth Electoral Roll is not completely consistent with the ABS sex standard.

    Interpretability
    Definition: Provides information on where to find more detail about the collection to help with interpretation and understanding.

    The results are simple counts and are easy to interpret. The ABS has published infographics, tables, and spreadsheets with explanatory notes that include details of the survey, the classifications used and a glossary of definitions.

    Accessibility
    Definition: Information on the ease of finding and acquiring data.

    Survey data are freely available on the ABS website. Information Line assistance is available.

    Integrity
    Recognising many in the community expected the ABS to operate a plebiscite-like process, the ABS included a number of special integrity measures in the conduct of the AMLPS:
      • independent assurance of quality and integrity controls
      • an 'external observer' program through which observers nominated by two ('Yes' and 'No') committees of Commonwealth parliamentarians reviewed the way ABS coded the marks on more than 600,000 survey form responses
      • a Fraud Control Plan and fraud control measures.

    Independent Assurance
    The ABS engaged global consulting firm Protiviti to provide independent quality assurance of AMLPS processes, and provided the following statement:

    "The ABS designed and implemented integrity controls over the end to end processes for managing and delivering the Australian Marriage Law Postal Survey (the Survey). These integrity controls included Integrity Gates at all critical points of the process.

    Protiviti provided assurance over the end to end processes for delivery of the Survey. This encompassed more than 30 Integrity Gates that spanned processes in the Survey from the initial mail despatch through to the compilation of statistical outputs.

    Protiviti provided assurance over the design of the Integrity Gates (whether the design of the Integrity Gate was appropriate to the task being checked and whether the checks were appropriate for the risks faced) and the application of the Integrity Gates (whether the checks were executed as intended). The assurance included consideration of the third parties supporting the end to end process. A key aspect of the independent assurance performed by Protiviti was the provision of formal sign off letter for each of the Integrity Gates.

    The approach adopted by the ABS was innovative, thorough and comprehensive and has supported the transparent and rigorous management of risks to the Survey."

    External observer program
    External observers, nominated by the Yes and No committees of Commonwealth Parliamentarians, were engaged to provide feedback on a key element of the survey process: the accurate coding of returned paper survey forms.

    The Commonwealth parliamentary committees were established solely for the purposes of the survey, one representing the Yes position, and one representing the No position. Each committee nominated around 60 external observers.

    Observers from both committees provided feedback on the coding of 606,991 survey returns which established that the accuracy of the survey return coding was extremely high with no bias in the capture of Yes or No responses.

    A full report on the External Observer Program is available.

    Fraud control
    The ABS put comprehensive measures in place to prevent, detect, respond to and report on potential fraud in the AMLPS.

    A designated Fraud Examiner was appointed to develop, implement and maintain a Fraud Control Plan to prevent, detect, assess, investigate, respond to and report on potential fraud in the AMLPS.

    The controls implemented to prevent fraud in the AMLPS included:
      • the introduction of the Marriage Law Survey (Additional Safeguards) Act 2017 which included provisions against fraud, specifically around the sale or purchase of survey forms
      • physical security of survey materials at all processing locations
      • use of secure access codes and single use 'mark in' codes to prevent counterfeiting of survey responses
      • reconciliation using the unique barcode on each survey form to ensure that only one response would be counted for each eligible respondent
      • cyber security controls including extensive use of data encryption
      • identity checks before issuing secure access codes and replacement survey forms
      • all staff involved in the AMLPS signed personal undertakings to remain impartial
      • all ABS staff were subject to the Marriage Law Survey (Additional Safeguards) Act 2017, the Census and Statistics Act 1905 and the APS Code of Conduct.

    In developing the Fraud Control Plan the ABS undertook a comprehensive risk assessment of all operations associated with the AMLPS.

    The Fraud Examiner worked closely with the Risk Manager throughout the process to monitor and review changing risk profiles in relation to fraud risks.

    In addition to fraud controls, the ABS took a proactive approach to managing fraud risks including:
      • responding quickly to resolve issues
      • engaging with online market providers to block and remove attempts to sell survey forms and responses
      • encouraging members of the public to report suspicious or criminal behaviour
      • raising awareness through the media of the serious penalties for fraud
      • guidance on the AMLPS website about the Safeguards Act and how to raise complaints
      • referring suspected cases of fraud to the relevant authorities.

    Throughout the survey period, issues reported to the ABS accounted for fewer than 500 individual survey forms (less than 0.0032% of over 16 million forms issued). These issues related to allegations of mail theft, survey forms offered for sale, persons influencing the response of vulnerable people, and the intention to respond to the survey on another person's behalf without their authorisation to do so.